Discovering Feature Weights for Feature-based Indexing of Q-tables

In this paper we propose an approach to address the old problem of identifying the feature conditions under which a gaming strategy can be effective. For doing this, we will build on previous work on CBRetaliate, a system that combines case-based reasoning and reinforcement learning to play team-based First Person Shooter Games. In CBRetaliate, cases are pairs (features, Q-table), where the Q-table associates a utility with each state-action pair, which is used to select an appropriate action in a given state. CBRetaliate learns cases as it plays against opponents. We propose to cluster cases in the case-base using a novel definition of similarity between their Q-tables; cases will be grouped in the same cluster if they have similar Q-tables. We propose to use standard information gain formulas and use the clusters as the classification to assign feature weights. We expect that this approach would lead to identifying features that are crucial to select which Q-table to reuse in a given situation. In addition, we propose to use the same notions of Qtable similarity to find substrategies that are common to every or nearly every case in the case base.